home
***
CD-ROM
|
disk
|
FTP
|
other
***
search
/
Tech Arsenal 1
/
Tech Arsenal (Arsenal Computer).ISO
/
tek-01
/
neuron.zip
/
NETWORK.DOC
< prev
next >
Wrap
Text File
|
1992-07-06
|
5KB
|
91 lines
*********************************************************************
NETWORK.DOC
*********************************************************************
VERSION 2.5 E.FARHI 07/92 TC 2.0
This is a short doc file for NETWORK.C
Read it carefully, it explains how to make and use a network.
You should have already read THEORY.DOC....
I shall tell you about local procedures and using NEURON.C .
*** Local procedures
print_output(net)
Prints the last layer state.
vector_modify(vector,size,high,low)
Modifies a vector containing some binary data (0 and 1) into a vector
containing low and high states, in order to send it to the net.
sort(vector,order,size)
Sorts a vector and sends back the sorting order.
print_input(Xmax,Ymax,input)
Prints the input as a Xmax*Ymax array (for caracters).
print_result(net)
Sorts and prints the network answer (output layer).
Gives the network identification of a caracter.
init_std_network(net)
Standart init of a net. If you don't want to think of how to initialize
a net...The network is then a multi-preceptron one.
learn(...)
Supervized learning using optimization algorithm.
verify(...)
Verifies that the net has a good memory.
*** How do I build my network.
First of all, you need to define your network size. For this initialize
nb_layers, nb_n_layer, nb_levels, layer_order. If you want to understand how
the layer order works, see NEURON.DOC.
Then, you need to set the prototypes. Each prototype is associated to
an output neuron (a neuron for each class). In the given example, the input isa
5*7 array, and the output, the alphabet.
You allocate memory and prepare the prototypes with vector_modify.
To teach the network, the only thing to do is to run 'learn_opt'.
Some disk utilities are available, and I even show how to recognize a
'strange' 'V' (not too strange). Finally, the network gives its answer...
*** Some advice. (see 'init_std_network')
I use a three precepton network because it can learn many things, and
computation is fast. The answering time is also quick. On the contrary, an
Hopfield net is slow (because it needs to stabilize) and can't memorize too
many things. A {0,1,2} order is fine.
The simulated tempering is useful when the net has difficulties to
learn, especially for Hopfield layers, because it enables to avoid interference
states created during the learning. You can decide of the tempering
temperatures and the exponential decreasing constant. If that constant is too
small, the learning time becomes very important.
The error threshold defines the learning depth. If it's high, the
learning is quick but the network doesn't know very well the prototypes at the
end. If it's low, learning time is longer, and the net has a good memory.
Anyway, it has been shown that a too small threshold doesn't fit well with the
generalization capacity. A net too specialized in what it has learnt can't
extrapolate its knowledge to unknown prototypes. That's why a 5 % to 10 % error
threshold is enough
The network temperature is a variable important. Finding an optimal
value is difficult. I use a value between 0.1 and 10 (why not 1), except during
tempering.
The improving variation coefficient for modifying links during
backpropagation can't be too high, else the network oscillates, even with the
shaker algorithm. Theorically, a high value decreases the learning time.. So
it's better too have a longer learning, but to be sure that the net will learn
something ! It depends of temperature. With beta=1/T=0.3 I use coef=0.3 .
If you use (as I do) continuum neurons, the high and low state can't be
reached because of the sigmoid shape (to reach those value, the potentials have
to be infinite, what leads to network saturation). So, for the learning
procedure, you have to impose some lower states in the outputs. That's why
coef_out is not 1 but 0.9 , then outputs are reachable.
The links and potential thresholds have to be used, to avoid the net
overflow. If the link threshold is low, the net can't memorize everything, and
will prefer to forget old prototypes, ad to learn the new ones. The bascule
threshold is generally 0 (else it causesa disymetry).
The random noise enables the net to avoid interference states. 5 % to
10 % is good.
the high and low states are usually +1 and -1. (0 and +1 is also
possible)
Anyway, to understand better this, you should have a look to the
'init_std_network' procedure in NETWORK.C .